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I.  INTRODUCTION 


Properly  managing  program  cost,  schedule,  and  performance  is  critical  to 
ensuring  that  programs  meet  their  intended  goals.  A  program  manager  is  tasked  with  the 
responsibility  of  tracking  planned  progress  versus  actual  progress  to  ensure  that 
customers  receive  value  for  the  time  and  money  expended  on  programs.  The  earned 
value  management  (EVM)  concept  is  a  project  management  tool  that  integrates  project 
scope  with  cost,  schedule,  and  perfonnance  elements  for  optimum  project  planning  and 
control.  The  Department  of  Defense  (DoD)  Earned  Value  Management  Division  states 
that  earned  value  management  should  be  fully  embraced  by  the  DoD  acquisition 
community  as  an  inherent  part  of  the  acquisition  program  management  value  chain 
because  it  provides  program  managers  accurate  and  timely  insight  into  cost,  schedule  and 
performance  of  DoD  weapons  systems  and  services  programs  (EVM,  2012).  Today’s 
shrinking  defense  budgets  focus  DoD  acquisitions  on  efficient  use  of  time  and  money  to 
give  the  best  solution  to  the  warfighter  and  the  taxpayer. 

EVM  has  been  used  by  DoD  to  forecast  program  cost  and  schedule  growth  over 
the  last  four  decades.  EVM  can  be  defined  as  a  project  management  technique  that 
focuses  on  the  planned  worked  accomplished  and  management’s  planned  budget  for  the 
work  for  the  purpose  of  monitoring  performance  and  predicting  the  final  required  costs 
and  the  time  necessary  to  finish  the  project.  EVM  has  been  used  by  DoD  since  the  1960s 
as  part  of  the  Cost/Schedule  Control  Systems  Criteria  (C\SCSC)  and  more  recently  in 
32  EVMS  guidelines  for  reporting  program  cost  and  schedule  infonnation.  These 
guidelines  have  been  adopted  by  the  American  National  Standards  Institute/Electronic 
Industries  Alliance  (ANSI/EIA-748  EVMS)  and  are  being  used  by  both  government  and 
industry.  EVM  tracks  the  cost  and  schedule  progress  and  reports  what  the  program  truly 
earned  for  the  budget  that  was  spent. 

EVM  is  required  on  all  defense  acquisition  program  cost  or  incentive  contracts 
over  $20  million  to  help  program  managers  track  perfonnance  and  predict  potential 
overruns  throughout  the  acquisition  life  cycle.  In  a  revision  to  the  DoD  earned  value 
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management  policy  memorandum  Michael  Wynne,  former  Under  Secretary  of  Defense 
Acquisition,  Technology  and  Logistics  (USD  (AT&L),  required  the  implementation  of 
EVM  methods  and  contract  perfonnance  reports  for  all  cost  or  incentive  contracts  valued 
at  or  greater  than  $20  million  in  then-year  dollars  and  it  was  incorporated  in  Department 
of  Defense  Instruction  (DoDI)  5000.02  (2008). 

EVM  has  been  used  by  acquisition  professionals  to  forecast  the  program’s 
schedule  performance  using  cost-based  indicators  but  does  not  completely  address  time- 
based  indicators.  Therefore,  even  though  EVM  has  been  used  since  the  early  1960s  as  a 
program  estimating  tool  it  is  viewed  by  some  professionals  as  incomplete  when 
predicting  schedule  performance  indicators.  An  extension  of  EVM,  called  earned 
schedule,  was  introduced  in  2003  as  a  tool  to  more  accurately  estimate  schedule 
performance  using  EVM  performance  indicators  that  were  lacking  in  traditional  EVM 
metrics. 

Earned  schedule  (ES)  was  developed  by  Walter  H.  Lipke  and  published  in  the 
March  2003  edition  of  the  Program  Management  Institute  College  of  Performance 
Management  Journal  (Lipke,  2003).  It  was  developed  to  provide  a  unique  method  to 
determine  schedule  performance  using  the  standard  EVM  performance  indicators  of 
budgeted  cost  of  work  scheduled  (BCWS),  actual  cost  of  work  performed  (ACWP), 
earned  value  (EV),  and  budget  at  completion  (BAC).  The  earned  schedule  concept 
identifies  the  time  at  which  the  amount  of  earned  value  (EV)  accrued  should  have  been 
earned  and  calculates  schedule  perfonnance  in  tenns  of  time,  not  costs. 

Earned  schedule  (ES)  uses  standard  EVM  performance  indicator  values  and  time- 
based  equations  to  depict  the  schedule  performance  and  may  help  a  program  manager 
estimate  schedule  perfonnance  more  accurately. 

A.  RESEARCH  SCOPE 

This  study  captures  earned  value  management  and  earned  schedule  data  on  four 
DoD  MDAPs.  It  compared  traditional  EVM  and  ES  performance  metrics  to  forecast  final 
program  duration. 
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The  data  collection  included  earned  value  data  on  four  U.S.  Army  Chemical 
Materials  Agency  (CMA)  programs.  The  programs  were  selected  based  on  their 
performance  measurement  baselines  and  milestone  similarities  but  due  to  the  complexity 
of  reporting  period  data  only  a  specific  portion  of  the  data  was  selected  for  this  research 
project.  Specifically,  data  was  collected  from  the  work  breakdown  structure  (WBS), 
operations  phase,  after  each  program  was  re -baselined  to  meet  a  Congressionally 
mandated  April  2012  completion  date.  After  program  re-baseline,  there  were  no  further 
schedule  changes  to  the  performance  measurement  baseline  (PMB)  for  the  operations 
phase.  Each  program  had  operations  phases  that  began  on  June  2007  and  completed  the 
incineration  or  neutralization  of  the  chemical  stockpile  within  a  five-year  period.  The 
similarities  between  the  operations  phase,  WBS  and  schedule  requirements  facilitated 
correlations  between  data  analysis  and  conclusions  across  the  programs. 

The  PMB  is  highly  important  to  earned  value  management  because  it  provides  the 
baseline  plan  to  measure  the  program’s  perfonnance.  It  is  the  sum  of  the  program’s 
planned  cost  over  time  and  establishes  the  scope,  schedule,  and  budget  for  a  program. 
The  earned  value  management  implementation  guide  (EVMIG)  describes  a  baseline  as 
having  the  following  characteristics:  it  accurately  represents  only  authorized  work  on  the 
contract,  it  includes  a  realistic  network  schedule  baseline,  and  it  includes  a  realistic  time 
phased  spread  of  budget/resources  to  the  baselined  schedule.  Additionally,  management 
makes  a  consistent  commitment  to  enforce  proper  baseline  change  procedures  and 
periodically  review  the  remaining  baseline  to  ensure  that  it  remains  executable  (2006).  A 
consistent  schedule  lent  itself  to  comparable  data  between  programs  and  was  essential  to 
answering  the  primary  research  question. 

B.  RESEARCH  OBJECTIVE 

The  objective  of  this  research  project  was  to  measure  the  accuracy  of  EVM  and 
ES  final  duration  forecast  metrics  on  U.S.  Army  CMA  programs.  The  traditional  EVM 
method  of  estimating  schedule  performance  uses  cost-based  indicators  and  Integrated 
Master  Schedule  (IMS)  (DoDI  5000.02,  2008)  to  represent  time.  But,  the  research  will 
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not  examine  the  DoD  EVM  practice  of  using  the  IMS  to  estimate  program  schedule.  The 
research  will  only  use  derived  mathematical  equations  to  measure  schedule  performance 
metrics  from  EV  data. 

The  research  will  detennine  if  the  earned  schedule  method  for  final  duration 
forecast  produces  a  more  accurate  measure  of  the  program’s  final  duration.  The  research 
paper  will  answer  the  following  questions: 

1.  Primary  Research  Question 

Is  the  ES  method  for  the  final  duration  forecast  more  accurate  than  EVM  methods 
for  final  duration  forecast  for  U.S.  Anny  CMA  programs? 

2.  Secondary  Research  Questions 

a)  How  do  EV  and  ES  final  duration  forecast  values  compare? 

b)  Can  the  ES  final  duration  forecast  be  easily  applied  to  DoD 
MDAPs? 

C.  METHODOLOGY 

The  research  project  methodology  was  developed  to  show  the  comparative  value 
of  EVM  and  ES  final  duration  forecast  methods  to  DoD  programs.  It  collected  traditional 
earned  value  management  indicator  data  on  four  similar  U.S.  Army  CMA  programs  and 
measured  final  duration  forecast  values  for  EVM  and  ES  methods.  The  study  used 
statistical  analysis  on  the  forecast  values  to  determine  the  variation  from  the  actual  final 
duration  over  time  and,  thus,  the  accuracy  of  the  methods. 

EVM  data  was  collected  on  four  U.S.  Army  CMA  programs  for  work  reported 
from  October  2007  until  the  completion  of  the  operations  phase  (munitions  destruction). 
The  earned  value  and  planned  value  cost-based  data  was  used  to  calculate  EVM  and  ES 
performance  metrics  and  produced  forecasts  for  final  program  duration. 
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A  spreadsheet  was  used  to  calculate  all  necessary  data  for  the  research  study.  An 
ES  spreadsheet  for  partial  time  periods  was  used  as  the  basis  for  the  data  analysis  and 
subsequently  modified  to  include  final  duration  forecast,  mean  squared  error,  and  percent 
difference  values. 

Final  duration  forecasts  were  determined  using  two  EVM  methods  and  one  ES 
method.  The  three  methods  were  the  EVM  planned  value  and  earned  value  methods,  and 
the  earned  schedule  method.  Respective  forecasts  for  each  reporting  period,  Mean 
Squared  Error  (MSE),  and  percent  differences  were  calculated  for  each  method. 

The  mean  squared  error  was  selected  because  it  reduced  all  forecast  values  to  data 
that  could  be  easily  analyzed  to  answer  the  primary  research  question.  It  showed  the 
statistical  variation  of  the  forecasted  final  duration  about  the  actual  duration  value  and 
was  used  to  measure  forecast  method  accuracy. 

This  research  project  calculated  MSE  at  program  percent  complete  periods  to 
measure  the  sensitivity  of  the  methods  to  time.  MSE  was  ranked  over  early  (10-40 
percent),  mid  (40-70  percent),  and  late  (70-100  percent)  stages  for  each  program. 
Smaller  MSE  values  showed  less  spread  in  the  data  and  therefore,  forecasted  values  that 
were  nearer  to  the  actual  final  duration. 

This  research  project  also  used  percent  difference  calculations  to  measure  the 
accuracy  of  the  final  forecast  value  compared  to  the  actual  value  over  time.  Percent 
difference  values  closer  to  zero  represented  forecasted  values  that  were  nearer  to  the 
actual  final  duration. 

D.  ORGANIZATION 

This  research  paper  is  organized  to  allow  for  a  logical  progression  from  objective 
to  conclusion  and  answering  the  primary  research  question:  Is  the  ES  method  for  the 
final  duration  forecast  more  accurate  than  EVM  methods  for  final  duration  forecast  for 
U.S.  Army  CMA  programs? 

The  INTRODUCTION  chapter  gives  an  expanded  version  of  the  research 
proposal.  It  establishes  the  scope,  objective,  and  methodology  of  the  research  paper. 
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The  BACKGROUND  chapter  introduces  the  concepts  of  earned  value 
management  and  earned  schedule.  It  specifically  addresses  the  key  schedule  performance 
metrics  and  applies  them  to  the  final  duration  forecasts. 

The  CASE  STUDY  AND  ANALYSIS  chapter  examines  four  U.S.  Army  CMA 
programs  using  earned  value  indicators  over  consistent  schedule  baselines.  It  documents 
the  traditional  EVM  and  ES  forecasted  final  duration  methods  to  measure  their  accuracy 
as  predictors.  An  accuracy  comparison  is  drawn  between  the  methods  to  help  answer  the 
primary  and  secondary  research  questions. 

The  SUMMARY  AND  RECOMMENDATIONS  chapter  provides  the  answer  to 
the  primary  research  question. 
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II.  BACKGROUND 


A.  EARNED  VALUE  MANAGEMENT  CONCEPT 

In  a  memorandum  for  acquisition  professionals,  The  Honorable  Dr.  Ashton  Carter 
(former  USD  (AT&L))  emphasized  the  need  to  deliver  better  value  to  the  taxpayer  and 
warfighter  by  improving  the  way  the  Department  of  Defense  does  business. 

We  must  therefore  abandon  inefficient  practices  accumulated  in  a  period 
of  budget  growth  and  learn  to  manage  defense  dollars  in  a  manner  this  is, 
to  quote  Secretary  Gates  at  his  May  8,  2010  speech  at  the  Eisenhower 
Library,  “respectful  of  the  American  taxpayer  at  a  time  of  economic  and 
fiscal  distress.”  (Carter,  2010) 

As  a  means  to  achieve  better  value,  cost  and  schedule  metrics  that  track  forecasted 
performance  became  key  DoD  program  management  tools.  Earned  value  management  is 
one  such  tool. 

Earned  value  management  is  a  widely  accepted  practice  for  project  management 
that  is  being  used  across  the  DoD,  the  Federal  government,  and  industry.  It  is  an 
integrated  management  system  that  coordinates  the  work  scope,  schedule,  and  cost  goals 
of  a  program  or  contract,  and  objectively  measures  progress  toward  these  goals.  EVM 
provides  a  prediction  of  the  final  costs  and  schedule  requirements  and  is  used  by  program 
managers  to: 

•  quantify  and  measure  program/contract  performance 

•  provide  an  early  warning  system  for  deviation  from  a  baseline 

•  identify  and  track  risks  associated  with  cost  and  schedule  overruns 

•  provide  a  means  to  forecast  final  cost  and  schedule  outcomes 

Earned  value  simply  allows  the  project  manager  to  take  advantage  of  their  actual 
results  and  forecast  the  final  results.  If  corrective  actions  are  taken  on  overruns  during 
the  early  stage  of  the  program  the  program  manager  may  change  the  forecasted  outcome. 
EVM  provides  the  program  manager  a  powerful  tool  in  the  successful  cost  and  schedule 
completion  of  a  program. 
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EVM  uses  performance  indicators  and  metrics  to  objectively  measure  program 
progress  against  the  budgeted  plan.  The  performance  indicators  provide  the  input  to 
measure  the  key  schedule  performance  metrics  used  in  EVM  analysis.  But,  by 
themselves,  earned  value  data  indicators  are  not  adequate  to  manage  the  project’s  time 
dimension  (Fleming  &  Koppehnan,  2010). 

1.  Performance  Indicators 

Figure  1  shows  notional  data  for  a  program  with  key  EVM  indicators  plotted, 
namely,  BCWS,  ACWP,  EV,  and  BAG.  The  EVM  indicators  are  defined  as  follows: 

•  BCWS,  also  called  planned  value  (PV),  is  the  ideal  time-phased  budget 
program  plan  for  work  currently  scheduled  or,  more  specifically,  how 
much  work  the  program  should  have  accomplished  at  a  specific  time  and 
cost. 

•  ACWP  is  the  actual  cost  incurred  while  accomplishing  the  program  work. 
The  ACWP  value  may  be  higher  or  lower  than  the  BCWP  at  a  specific 
time  increment  depending  on  the  program’s  progress. 

•  EV  is  the  budgeted  cost  for  work  performed  (BCWP)  on  a  program.  It  is 
the  value  of  the  completed  work  in  tenns  of  the  work’s  assigned  budget, 
not  simply  what  has  been  spent  on  the  program. 

•  BAC  is  budgeted  cost  at  completion. 

A  monthly  contract  performance  report  (CPR)  is  submitted  to  the  government 
program  manager  with  detailed  perfonnance  indicator  data  (Data  Item  Description  (DID) 
DI-MGMT-81466A,  2005)  to  monitor  and  measure  EVM  metrics. 
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Figure  1.  EVM  Indicators 


2.  Schedule  Performance  Index 

An  important  schedule  metric  that  can  be  derived  from  the  EVM  indicators  is  the 
program’s  schedule  performance  index.  The  SPI  helps  predict  the  duration  of  the 
planned  work  by  detennining  if  the  project  is  ahead,  on,  or  behind  schedule  at  any  point 
in  time  on  the  BCWS  line.  It  is  calculated  with  cost-based  data  and  can  be  an  early 
indicator  of  program  schedule  problems.  It  is  a  measure  of  the  program’s  schedule 
efficiency  and  is  calculated  by  taking  the  ratio  of  the  earned  value  to  the  BCWS.  The 
following  equation  was  used  to  calculate  SPI: 


SPI  = 


EV 

BCWS  ' 
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The  ratio  yields  a  larger  value  when  more  work  was  performed  than  the  planned 
value.  Conversely,  the  ratio  yields  a  smaller  value  when  less  work  was  performed  than 
the  planned  value.  Favorable,  or  schedule  efficient  programs  have  a  SPI  value  greater 
than  1.0  and  inefficient  programs  have  a  SPI  value  less  than  1.0. 

Figure  2  shows  notional  SPI  data  for  a  program  that  completed  behind  schedule. 
The  data  represents  a  program  that  was  scheduled  for  a  12-month  completion  date,  but 
finished  in  15  months.  It  shows  baseline  and  actual  SPI  values  plotted  on  the  graph. 
Baseline  SPI  values  show  an  ideal  value  throughout  the  program’s  planned  schedule  and 
the  SPI  line  shows  the  program’s  actual  schedule  efficiency.  The  program’s  SPI  values 
fluctuate  depending  on  the  amount  of  planned  work  budgeted  compared  to  the  earned 
value  of  the  work. 

The  Figure  2  program  had  poor  schedule  performance  and  indicated  that  the 
program  may  not  complete  within  the  12-month  planned  schedule.  The  SPI  values  were 
less  than  1.0  until  program  completion  and  did  not  start  to  improve  until  month  10.  At 
12  months,  the  program  had  not  completed  as  planned  but  continued  to  improve  until  it 
completed  at  month  15. 
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Figure  2.  SPI  for  Late  Completing  Program 

Using  traditional  earned  value  management  methods,  actual  SPI  values  will 
naturally  reach  an  ideal  value  of  1.0  at  program  completion  because  SPI  values  are 
calculated  with  costs.  EVM  calculates  schedule  efficiency  based  on  budgeted  costs  and 
therefore  may  not  accurately  reflect  time-base  schedule  efficiency.  At  program 
completion  all  funds  are  expended  and  the  SPI  ratio  always  equals  1.0.  Based  on  costs,  at 
program  completion  all  programs  whether  early,  on  time,  or  late,  show  a  favorable 
schedule  performance.  Program  managers  must  analyze  the  SPI  trend  and  not  just  focus 
on  the  later  program  stage  values  because  they  may  give  unreliable  indications  of 
schedule  performance  when  assessing  risks  and  corrective  actions. 
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3. 


Final  Duration  Forecast 


Another  important  metric  related  to  schedule  performance  that  can  be  derived 
from  the  EVM  indicators  is  the  independent  estimate  at  completion  (IEAC(t))  (Lipke, 
2009a).  It  predicts  the  program  completion  time  based  on  elapsed  time  and  forecast  of 
work  remaining  on  the  program.  The  commonly  applied  form  for  IEAC(t)  equals  the 
elapsed  time  plus  the  forecast  for  work  remaining.  It  can  be  expressed  as  the  following 
formula: 


where 


IEAC(t)  =  AT  + 


BAC-EV 

WR 


AT  =  actual  time  when  PV  and  EV  are  reported 
WR  =  work  rate  factor  that  converts  work  into  time. 


The  two  common  work  rates  that  are  applied  to  forecast  final  duration  are  average 
BCWS  (BCWSav)  and  average  earned  value  (EVav).  BCWSav  value  is  used  to  depict 
program  performance  that  is  expected  to  progress  according  to  plan.  EVav  is  used  to 
depict  program  performance  that  is  expected  to  follow  the  current  SPI  trend.  The  two 
work  rates  are  shown  in  the  following  equations  and  were  used  to  calculate  the 
IEAC(t)Bcws  and  IEAC(t)EV  values: 


where 


BCWS.  „  = 


BCWS. 


n 


EV 

JC  J/  _  cum 


BCWSCum  =  cumulative  value  of  BCWS 
EV cum  =  cumulative  value  of  EV 
n  =  total  number  of  time  increments  within  AT. 
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EVM  methods  for  forecasting  project  duration  have  been  taught  in  training 
courses  and  used  by  project  managers  for  four  decades  (Lipke,  2009a).  But,  EVM  cost- 
based  indicator  values  may  not  always  accurately  predict  time.  So,  the  earned  schedule 
method  was  developed  to  measure  schedule  performance  using  standard  EVM  indicators 
and  time. 

B.  EARNED  SCHEDULE  CONCEPT 

In  March  2003,  Walter  Lipke  published  a  paper  in  The  Measurable  News  that 
introduced  an  extension  of  earned  value  management  that  tracked  program  schedule  in 
units  of  time  rather  than  traditional  EVM  units  of  budget,  called  earned  schedule  (Lipke, 
2003).  It  was  developed  in  response  to  the  noted  deficiency  in  using  EVM  cost-based 
indicators  to  effectively  evaluate  program  schedule  perfonnance.  Schedule  performance 
is  important  because  if  a  product  is  not  delivered  on  time  there  can  be  serious 
repercussions.  In  addition  to  the  likelihood  of  increased  project  costs,  the  customer, 
internal  or  external,  is  deprived  of  using  the  product,  consequently  preventing  the 
delivery  of  their  product  or  service  (Lipke,  2009a). 

The  basis  for  the  earned  schedule  concept  is  straightforward.  Identify  the  time 
at  which  the  amount  of  earned  value  (EV)  accrued  should  have  been  earned.  By 
determining  this  time,  time-based  indicators  can  be  formed  to  provide  schedule  variance 
and  perfonnance  efficiency  management  information  (Earned  Schedule,  2012). 

ES  also  uses  perfonnance  indicators  and  metrics  to  objectively  measure  program 
progress  against  the  budgeted  plan.  The  performance  indicators  provide  the  input  to 
calculate  the  key  schedule  performance  metrics  used  in  ES  analysis.  ES  can  also  provide 
schedule  performance  indicators  that  aide  a  program  manager  in  the  successful 
completion  of  a  program. 

1.  Performance  Indicators 

From  the  time  of  the  development  of  EVM  indicators,  it  has  been  known  that  the 
schedule  performance  metrics  are  flawed  and  exhibit  strange  behavior  over  the  final  third 

of  a  project,  when  perfonnance  is  poor  (Lipke,  2003).  As  a  program  nears  completion, 
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the  BCWS  and  BCWP  naturally  approach  each  other  so  that  all  scheduled  budgets  have 
been  used  and  the  program  theoretically  completed  on  time  using  EVM  schedule 
performance  metrics.  The  standard  EVM  performance  indicator  data  included  in  the 
CPR  only  represents  budget  data  and  any  schedule  metrics  calculations  from  it  will  not 
measure  time.  Therefore,  EVM  schedule  performance  metrics  have  not  always  been 
viewed  by  project  mangers  as  being  reliable  indicators  for  schedule  (Lipke,  2003). 

Figure  3  shows  notional  data  for  an  acquisition  program  with  EV  and  BCWS 
plotted  on  the  graph.  The  new  value,  earned  schedule,  is  calculated  by  projecting  the  EV 
data  point  onto  the  BCWS  curve  to  determine  where  EV  equals  the  planned  value 
(BCWS)  for  the  program.  Graphically,  the  horizontal  dashed  line  projection  from  the 
EVt  curve  to  the  BCWS  curve  identifies  the  time  that  amount  of  EV  should  have  been 
earned  in  accordance  with  the  schedule.  Extending  a  vertical  dashed  line  from  the 
BCWS  curve  to  the  x-  axis  yields  a  time  increment  for  the  earned  schedule  value. 
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Figure  3. 
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The  earned  schedule  principle  can  be  graphically  explained  but  an  equation  is 
needed  to  accurately  calculate  the  ES  values.  The  equation  uses  actual  EV  and  BCWS 
data  values  that  are  supplied  in  monthly  CPRs.  The  following  equation  uses  linear 
interpolation  to  calculate  the  earned  schedule: 


ES  =  t  + 


EVi+2-BCWS, 
yBCWSt+1-BCWSt , 


The  linear  interpolation  method  that  was  used  to  detennine  ES  is  represented  in 
Figure  4. 


BCWS  t+2 


Figure  4.  Linear  Interpolation  Method  to  Calculate  ES  (From  Fast,  2012) 

Using  t=8,  EVt+2=2300,  BCWSt=2135,  BCWSt+i=2435  from  Figure  3  data  yields 
a  value  of  8.55  months,  which  indicates  that  the  program  was  behind  schedule  because  it 
took  10  months  to  accomplish  work  that  was  only  planned  for  8.55  months. 
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ES  =  8  + 


2300-2135 


=  8.55 


f 


V  2435-2135  J 


2.  Schedule  Performance  Index 


The  important  perfonnance  metric  that  can  be  derived  from  the  ES  is  the 
program’s  schedule  performance  index,  SPI(t).  It  is  a  representation  of  how  efficiently  a 
program  is  performing  to  schedule.  Lipke  derived  it  from  the  ratio  between  actual  time 
expended  (ATE)  and  earned  schedule.  The  following  equation  was  used  to  calculate 
SPI(t): 


SPI(t)  = 


ES 

ATE  ' 


Just  as  in  traditional  EVM  schedule  performance  metrics,  an  SPI(t)  value  greater 
than  1.0  is  favorable  and  a  value  less  than  1.0  is  unfavorable. 

Figure  5  shows  the  notational  SPI(t)  data  for  an  acquisition  program  that 
completed  behind  schedule.  The  data  represents  a  program  that  was  scheduled  for  a  12- 
month  completion  date,  but  finished  in  15  months.  Just  as  in  traditional  EVM,  the 
baseline  SPI  values  equaled  1.0  throughout  the  program’s  planned  schedule.  But,  the 
actual  SPI(t)  lines  showed  different  values  than  the  SPI  graph  from  Figure  2.  The 
program  performed  poorly  throughout  its’  duration  and  it  was  clear  by  the  SPI(t)  values 
that  the  program  was  in  danger  of  not  completing  on  time.  All  SPI(t)  values  are  less  than 
1.0  and  after  the  planned  12-month  completion  date  they  did  not  improve  as  the  program 
reached  completion  because  the  program  completed  behind  schedule.  The  critical 
difference  between  EVM  and  ES  schedule  performance  index  metrics  is  time. 
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Figure  5.  SPI(t)  for  Late  Completing  Program 

SPI(t)  calculations  factor  in  the  time  that  the  earned  value  should  have  been 
accomplished  and  the  actual  time  it  was  accomplished.  Therefore,  SPI(t)  does  not 
necessarily  reach  favorable  numbers  for  all  programs  because  costs  are  not  the  primary 
variables.  ES  metrics  yield  values  that  vary  depending  on  the  schedule  efficiency 
measured  in  time. 

3.  Final  Duration  Forecast 

Just  as  with  EVM,  earned  schedule  perfonnance  metrics  can  also  be  used  to 
forecast  final  duration.  The  major  difference  between  the  two  methods  is  that  ES 
incorporates  time  with  the  planned  duration  (PD)  and  SPI(t)  values  in  the  equation. 
Lipke  derived  the  following  equation  to  calculate  ES  final  duration  forecast  (IEAC(t)Es): 
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IEAC(t)F o  = -  . 

s  SPI(t ) 

A  simulation  study  published  in  The  Measurable  News  journal  compared  EVM 
and  ES  duration  forecast  methods  and  concluded  that  the  ES  metrics  outperform,  on  the 
average,  EVM  duration  forecast  methods.  Also,  the  ES  method  is  more  reliable  in  ah 
stages  (early  stage,  mid  stage,  late  stage)  of  the  project  life  cycle  (Vanhouke,  2007). 

C.  STATISTICAL  ANALYSIS 


Statistical  methods  were  used  to  determine  the  difference  between  estimated 
values  (forecasted  final  duration)  and  true  values  (actual  final  duration)  and  indicate  the 
accuracy  of  the  forecast.  Lipke  used  the  following  equation,  termed  standard  deviation, 
to  detennine  the  relative  accuracy  of  the  forecasted  value  about  the  actual  value  in  his 
research  (Lipke,  2009b): 


X(/fMC(0,„(0-^)2 


Standard  Deviation  = 


n  —  1 


Lipke’ s  equation  attempted  to  determine  the  variance  of  the  forecasted  values 
about  the  actual  value  at  each  time  increment.  But,  a  major  difference  between  Lipke’s 
equation  and  the  common  standard  deviation  equation  is  a  mean  value.  Standard 
deviation  calculates  deviation  about  a  mean  value  and  Lipke’s  equation  does  not  use  a 
mean  value,  rather,  it  uses  the  fixed  final  duration  value.  This  variation  from  the 
common  standard  deviation  equation  led  the  author  to  examine  an  alternate  statistical 
equation  for  this  paper. 

The  author  chose  the  mean  squared  error  equation  to  measure  the  variance  in  the 
forecasted  value  about  the  actual  value: 


MSB  =  -(^{IEACit),  -FD)f 


i= 1 


where 


IEAC(t);  =  independent  estimate  at  completion  for  each  reporting  period 
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FD  =  actual  final  duration 


n  =  number  of  reporting  periods  (entire  known  population). 

The  MSE  and  Lipke’s  standard  deviation  equation  are  very  similar  to  each  other. 
Both  measure  the  squared  value  of  the  summation  of  the  difference  between  IEAC(t)  and 
FD  divided  by  a  number  of  time  increments.  In  Lipke’s  equation,  the  square  root  of  the 
summation  squared  is  shown  to  equal  the  standard  deviation.  Statistically,  the  squared 
root  of  the  variance  equals  the  standard  deviation.  The  author  chose  MSE  to  show  the 
variance  in  the  forecast  about  the  actual  value  but,  taking  the  square  root  of  MSE  yields 
the  root  mean  square  deviation  (RMSD),  also  known  as  standard  deviation.  Therefore, 
the  RMSD  is  analogous  to  Lipke’s  standard  deviation  equation  and  the  author  believes 
that  MSE  accurately  measures  the  error  between  the  forecasted  and  actual  value  while 
still  maintaining  the  basis  of  Lipke’s  research. 

A  lower  MSE  indicated  that  the  forecasted  duration  was  closer  to  the  actual 
duration,  whereas  a  higher  MSE  indicated  that  the  forecasted  duration  was  further  from 
the  actual  duration.  The  goal  of  this  equation  is  to  show  an  IEAC(t)  method  that  is 
closest  to  the  actual  duration. 


The  author  also  used  the  percent  difference  equation  to  determine  the  difference 
in  the  forecasted  value  versus  the  actual  value  and  measure  forecasted  value  accuracy. 
Percent  difference  was  calculated  with  the  following  equation: 


Percent  Difference  = 


,EAC«)-FDxm  . 

FD 


The  percent  difference  value  must  remain  small  enough  throughout  the  program 
as  to  not  negatively  affect  the  schedule  because  too  large  a  difference  would  produce  a 
Nunn-McCurdy  Act  (1983,  §  2433)  significant  schedule  breach  indication. 

Table  1  summarizes  the  statistical  analysis  results  as  applied  to  the  forecasted 
final  duration  analysis. 
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Table  1 .  Interpretation  of  Statistical  Analysis  Values 


Mean  Squared  Error 

Percent  Difference 

IEAC(t)  Accuracy 

Low 

Low 

High 

High 

High 

Low 

D.  SUMMARY  OF  THE  EVM  AND  ES  PERFORMANCE  METRICS 

The  ES  method  for  SPI(t)  and  final  duration  forecast  showed  advantages  over  the 
EVM  method  for  SPI  and  final  duration  forecast.  ES  used  time  indicators  in  the  schedule 
performance  equations,  rather  than  the  EVM  cost  indicators,  which  may  more  clearly 
represent  program  time  against  the  PMB.  ES  values  were  also  easily  measured  with 
existing  EVM  performance  indicator  data  and  the  schedule  performance  calculations  only 
required  known  program  times  or  dates.  Table  2  shows  a  comparison  between  EVM  and 
ES  performance  equations. 

Table  2.  Summary  of  EVM  and  ES  Performance  Metric  Equations 


Method 

SPI 

IEAC(t) 

EVM 

EV 

(BAC-EV) 

BCWS 

BCWSm 

AT  +  (BAC-EV) 

EV 

av 

ES 

ES 

PD 

ATE 

SPI(t ) 
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III.  CASE  STUDY  AND  ANALYSIS 


A.  INTRODUCTION 

In  November  1985,  Public  Law  99-145,  Section  1142  (50  United  States  Code, 
Section  1521)  was  introduced  and  required  the  safe  destruction  of  the  US  military 
chemical  weapons  stockpile.  It  directed  DoD  to  dispose  of  the  lethal  unitary  chemical 
agents  and  munitions  stored  at  eight  Army  installations  and  required  disposal  facilities  to 
be  cleaned,  dismantled,  and  disposed  of  according  to  applicable  laws  and  regulations. 

U.S.  Army  CMA  was  created  to  incorporate  the  former  Program  Manager  for 
Chemical  Demilitarization  and  portions  of  the  U.S.  Army  Soldier  and  Biological 
Chemical  Command  into  one  agency.  The  agency’s  headquarters,  scientific, 
communications,  and  support  staff  are  located  at  the  Edgewood  area  of  the  Aberdeen 
Proving  Ground,  MD. 

The  U.S.  Anny  CMA  is  the  world  leader  in  programs  to  store,  treat,  and  dispose 
of  chemical  weapons  safely  and  effectively.  The  agency  developed  and  used  incineration 
and  neutralization  technologies  to  safely  store  and  eliminate  chemical  weapons  at  seven 
stockpile  sites  while  protecting  the  public,  its  workers  and  the  environment.  The  U.S. 
Army  CMA  fulfilled  its  mission  and  safely  destroyed  the  Nation’s  aging  chemical 
weapons  using  incineration  and  neutralization  technologies. 

The  U.S.  Army  CMA  programs  exceeded  ten  years  from  initial  facility 
construction  to  complete  facility  closure  and  were  funded  in  excess  of  $50  million  each. 
The  contracts  were  incentive  type  so  they  were  required  to  follow  DoD  EVM  policy  in 
accordance  with  DoDI  5000.02  (2008). 

In  a  revision  to  the  DoD  EVM  policy,  Michael  W.  Wynne,  former  USD  (AT&L), 
required  programs  valued  at  or  greater  than  $50  million  in  then-year  dollars  follow  the 
thirty-two  management  guidelines  published  in  ANSI/EIA-748  EVMS,  submit  a  monthly 
CPR,  and  a  monthly  IMS. 
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The  four  U.S.  Army  CMA  programs  selected  for  this  research  followed  the  DoD 
EVM  policy  requirements  and  provided  timely  and  consistent  EVM  data  to  the  program 
manager.  The  programs  were  selected  for  this  research  paper  based  on  their  common 
EVM  data,  WBS,  and  schedule  characteristics.  Specifically,  data  was  collected  from  the 
WBS  level  for  the  operations  phase  (agent  destruction)  after  each  program  was  re- 
baselined  to  meet  a  Congress  mandated  April  2012  completion  date.  The  operations  were 
conducted  in  the  same  operations  phase  WBS  level  for  each  program. 

Each  program  had  a  single  prime  contractor  that  managed  its  validated  EVMS 
with  one  or  two  subcontractors  that  provided  EVM  data  to  the  prime  contractor.  The 
subcontractors  were  fully  integrated  into  prime  contractor  so  there  was  only  one  EVMS 
per  program. 

Each  program  was  re-baselined  and  it  was  a  significant  event.  The  prime 
contractors  were  offered  large  guaranteed  monetary  incentives  to  complete  the  operations 
phase  within  schedule.  After  the  program  re-baseline  on  June  30,  2007  there  were  no 
further  changes  to  the  schedule  baseline  for  the  operations  phase  and  each  was  expected 
to  complete  by  April  30,  2012.  All  four  U.S.  Army  CMA  programs  had  the  same  start 
date  and  same  planned  completion  date,  which  facilitated  correlations  between  data 
analysis  and  conclusions  across  the  programs. 

Due  to  the  sensitivity  of  the  data  no  actual  program  names  or  contractors  will  be 
discussed  in  this  research  paper.  The  programs  will  be  designated  Program  A,  Program 
B,  Program  C,  and  Program  D  and  all  data  and  analysis  will  be  anonymous.  Further 
details  on  the  programs  and  EV  data  must  be  coordinated  through  the  U.S.  Army  CMA 
headquarters. 

B.  PROGRAM  A 

EVM  data  was  collected  for  the  operation  phase  (BAC  =  $207,289,000)  over  the 
reporting  period  30  June  2007  to  30  September  2011  and  used  to  measure  schedule 
performance  metrics. 
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Figure  6  shows  the  MSE  of  the  forecasted  final  duration  for  the  three  different 
IEAC(t)  methods.  The  MSE  for  each  of  the  IEAC(t)  methods  converged  near  the  actual 
duration  throughout  the  program  and  are  consistent  with  any  type  of  forecast  method. 

The  ES  MSE  plot  was  significantly  lower  than  the  BCWS  and  EV  plots 
throughout  the  program.  The  ES  MSE  plot  confirms  that  the  ES  method  gave  a  closer 
final  duration  forecast  than  the  BCWS  and  EV  methods.  But,  the  plots  only  show  that 
IEAC(t)  ES  was  better  relative  to  the  BCWS  and  EV  methods  and  does  not  indicate  the 
true  accuracy  of  the  data.  For  example,  during  the  early  program  stage  the  MSE  values 
were  at  least  110  months  squared  for  every  reporting  period.  A  MSE  =110  translated  into 
a  deviation  of  at  least  10  months  from  the  actual  duration. 

The  percent  of  total  program  duration  was  calculated  by  dividing  the  forecasted 
deviation  months  with  the  total  program  duration  months.  The  calculated  value  gave  a 
clear  indication  of  the  relative  accuracy  of  the  forecasted  value  compared  to  the  actual 
length  (time)  of  the  program.  In  the  case  of  Program  A,  the  forecasted  value  deviation 
was  20  percent  of  the  actual  program  duration,  which  would  not  have  been  useful  to  a 
program  manager  for  schedule  planning  because  it  would  have  signaled  a  false  significant 
schedule  breach  indicator.  Though  lower  than  EV  and  BCWS  methods,  the  IEAC(t)  ES 
method  would  still  not  be  helpful  to  a  program  manager  because  it  showed  Nunn- 
McCurdy  breach  indicators  that  were  false. 
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Figure  6. 


Mean  Squared  Error  Results  for  Final  Duration  Forecast-Program  A 


Figure  7  shows  the  percent  difference  of  the  forecasted  duration  (IEAC(t))  from 
the  actual  duration  for  each  of  the  three  methods  plotted  over  time.  Again,  all  three 
methods  converged  near  the  actual  final  duration  as  time  progresses  because  the  closer 
the  program  gets  to  completion  the  more  confidence  the  method  had  in  predicting  the 
actual  duration.  But,  none  of  the  IEAC(t)  methods  showed  a  high  confidence  in  helping 
the  program  manager  forecast  events.  All  IEAC(t)  method  values  were  at  least 
25  percent  over  the  actual  duration  throughout  the  majority  of  the  program.  Specifically, 
forecast  methods  at  25  percent  different  from  the  actual  duration  translate  into  missing 
the  end  date  by  at  least  13  months.  That  type  of  inaccuracy  in  the  forecast  values  would 
immediately  lead  to  a  Nunn-McCurdy  Act  schedule  breach  indicator.  But,  history  shows 
that  after  program  re-baseline  the  program  did  not  experience  a  breach  and  completed 
ahead  of  schedule.  The  data  showed  that  acceptable  values,  less  than  11  percent 
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difference  to  prevent  a  Nunn-McCurdy  breach  indicator,  were  generally  not  achieved 
until  the  very  late  stages  of  the  program  where  it  is  easier  to  make  accurate  forecasts. 
Though  ES  IEAC(t)  showed  improvement  over  the  BCWS  and  EV  IEAC(t)  methods  it 
would  still  not  give  a  program  manager  confidence  in  forecasting  schedule  perfonnance. 
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Figure  7.  Percent  Difference  Results  for  Final  Duration  Forecast-Program  A 


C.  PROGRAM  B 

EVM  data  was  collected  for  the  operation  phase  (BAC  =  $377,521,000)  over  the 
reporting  period  30  June  2007  to  29  January  2012  and  used  to  measure  schedule 
performance  metrics. 

Figure  8  shows  the  MSE  of  the  forecasted  final  duration  for  the  three  different 
IEAC(t)  methods.  The  MSE  for  each  of  the  IEAC(t)  methods  converged  near  the  actual 
duration  throughout  the  program  and  are  consistent  with  any  type  of  forecast  method. 
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The  ES  MSE  plot  was  significantly  lower  than  the  BCWS  and  EV  plots 
throughout  the  early  and  mid  program  stages.  The  ES  MSE  plot  confirms  that  the  ES 
method  gave  a  closer  final  duration  forecast  than  the  BCWS  and  EV  methods.  But,  the 
plots  only  show  that  IEAC(t)  ES  was  better  relative  to  the  BCWS  and  EV  methods  and 
does  not  indicate  the  true  accuracy  of  the  data.  For  example,  during  the  early  program 
stage  the  MSE  values  were  at  least  60  months  squared  for  every  reporting  period. 
A  MSE  =60  translated  into  a  deviation  of  at  least  8  months  from  the  actual  duration.  In 
the  case  of  Program  B,  the  forecasted  value  deviation  was  15  percent  of  the  actual 
program  duration,  which  would  not  have  been  useful  to  a  program  manager  for  schedule 
planning  because  it  would  have  signaled  a  false  significant  schedule  breach  indicator. 
Though  lower  than  EV  and  BCWS  methods,  the  IEAC(t)  ES  method  would  still  not  be 
helpful  to  a  program  manager  because  it  showed  Nunn-McCurdy  breach  indicators  that 
were  false. 
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Figure  8. 


Mean  Squared  Error  Results  for  Final  Duration  Forecast-Program  B 
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Figure  9  shows  the  percent  difference  of  the  forecasted  duration  (IEAC(t))  from 
the  actual  duration  for  each  of  the  three  methods  plotted  over  time.  None  of  the  IEAC(t) 
methods  showed  a  high  confidence  in  helping  the  program  manager  forecast  events.  All 
IEAC(t)  method  values  were  at  least  15  percent  over  the  actual  duration  throughout  the 
majority  of  the  program.  Specifically,  forecast  methods  at  15  percent  different  from  the 
actual  duration  translate  into  missing  the  end  date  by  at  least  9  months.  That  type  of 
inaccuracy  in  the  forecast  values  would  immediately  lead  to  a  Nunn-McCurdy  Act 
schedule  breach  indicator.  But,  history  shows  that  after  program  re-baseline  the  program 
did  not  experience  a  breach  and  completed  ahead  of  schedule.  The  data  showed  that 
acceptable  values,  less  than  10  percent  difference,  to  prevent  a  Nunn-McCurdy  breach 
indicator  were  generally  not  achieved  until  the  very  late  stages  of  the  program  where  it  is 
easier  to  make  accurate  forecasts.  Though  ES  IEAC(t)  showed  improvement  over  the 
BCWS  and  EV  IEAC(t)  methods  would  still  not  give  a  program  manager  confidence  in 
forecasting  schedule  performance. 
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Figure  9.  Percent  Difference  Results  for  Final  Duration  Forecast-Program  B 


D.  PROGRAM  C 

EVM  data  was  collected  for  the  operation  phase  (BAG  =  $175,1 12,000)  over  the 
reporting  period  30  June  2007  to  28  October  2011  and  used  to  measure  schedule 
performance  metrics. 

Figure  10  shows  the  MSE  of  the  forecasted  final  duration  for  the  three  different 
IEAC(t)  methods.  The  MSE  for  each  of  the  IEAC(t)  methods  converged  near  the  actual 
duration  throughout  the  program  and  are  consistent  with  any  type  of  forecast  method. 

The  ES  MSE  plot  was  significantly  lower  than  the  BCWS  and  EV  plots 
throughout  the  program.  The  ES  MSE  plot  confirms  that  the  ES  method  gave  a  closer 
final  duration  forecast  than  the  BCWS  and  EV  methods.  But,  the  plots  only  show  that 

lEAC(t)  ES  was  better  relative  to  the  BCWS  and  EV  methods  and  does  not  indicate  the 
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true  accuracy  of  the  data.  For  example,  during  the  early  program  stage  the  MSE  values 
were  at  least  57  months  squared  for  every  reporting  period.  A  MSE  =57  translated  into  a 
deviation  of  at  least  8  months  from  the  actual  duration.  In  the  case  of  Program  C,  the 
forecasted  value  deviation  was  15  percent  of  the  actual  program  duration,  which  would 
not  have  been  useful  to  a  program  manager  for  schedule  planning  because  it  would  have 
signaled  a  false  significant  schedule  breach  indicator.  Though  lower  than  EV  and 
BCWS  methods,  the  IEAC(t)  ES  method  would  still  not  be  helpful  to  a  program  manager 
because  it  showed  Nunn-McCurdy  breach  indicators  that  were  false. 
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Figure  10. 


Mean  Squared  Error  Results  for  Final  Duration  Forecast-Program  C 


Figure  1 1  shows  the  percent  difference  of  the  forecasted  duration  (IEAC(t))  from 
the  actual  duration  for  each  of  the  three  methods  plotted  over  time.  Again,  all  three 
methods  converged  near  the  actual  final  duration  as  time  progresses  because  the  closer 
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the  program  gets  to  completion  the  more  confidence  the  method  had  in  predicting  the 
actual  duration.  But,  none  of  the  IEAC(t)  methods  showed  a  high  confidence  in  helping 
the  program  manager  forecast  events.  All  IEAC(t)  method  values  were  at  least 
30  percent  over  the  actual  duration  throughout  the  majority  of  the  program.  Specifically, 
forecast  methods  at  30  percent  different  from  the  actual  duration  translate  into  missing 
the  end  date  by  at  least  16  months.  That  type  of  inaccuracy  in  the  forecast  values  would 
immediately  lead  to  a  Nunn-McCurdy  Act  schedule  breach  indicator.  But,  history  shows 
that  after  program  re-baseline  the  program  did  not  experience  a  breach  and  completed 
ahead  of  schedule.  The  data  showed  that  acceptable  values,  less  than  1 1  percent 
difference,  to  prevent  a  Nunn-McCurdy  breach  indicator  were  generally  not  achieved 
until  the  very  late  stages  of  the  program  where  it  is  easier  to  make  accurate  forecasts. 
Though  ES  IEAC(t)  showed  improvement  over  the  BCWS  and  EV  IEAC(t)  methods 
would  still  not  give  a  program  manager  confidence  in  forecasting  schedule  perfonnance. 


- IEAC(t)ES 

- IEAC(t)BCWS 

—  -IEAC(t)EV 


Figure  1 1 .  Percent  Difference  Results  for  Final  Duration  Forecast-Program  C 
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E.  PROGRAM  D 

EVM  data  was  collected  for  the  operation  phase  (BAG  =  $1 13,232,000)  over  the 
reporting  period  30  June  2007  to  26  November  2010  and  used  to  measure  schedule 
performance  metrics. 

Figure  12  shows  the  MSE  of  the  forecasted  final  duration  for  the  three  different 
IEAC(t)  methods.  The  MSE  for  each  of  the  IEAC(t)  methods  converged  near  the  actual 
duration  throughout  the  program  and  are  consistent  with  any  type  of  forecast  method. 

The  ES  MSE  plot  was  significantly  lower  than  the  BCWS  and  EV  plots 
throughout  the  program.  The  ES  MSE  plot  confirms  that  the  ES  method  gave  a  closer 
final  duration  forecast  than  the  BCWS  and  EV  methods.  But,  the  plots  only  show  that 
IEAC(t)  ES  was  better  relative  to  the  BCWS  and  EV  methods  and  does  not  indicate  the 
true  accuracy  of  the  data.  For  example,  during  the  early  program  stage,  the  MSE  values 
were  at  least  119  months  squared  for  every  reporting  period.  A  MSE  =119  translated  into 
a  deviation  of  at  least  10  months  from  the  actual  duration.  In  the  case  of  Program  D,  the 
forecasted  value  deviation  was  24  percent  of  the  actual  program  duration,  which  would 
not  have  been  useful  to  a  program  manager  for  schedule  planning  because  it  would  have 
signaled  a  false  significant  schedule  breach  indicator.  Though  lower  than  EV  and  BCWS 
methods,  the  IEAC(t)  ES  method  would  still  not  be  helpful  to  a  program  manager 
because  it  showed  Nunn-McCurdy  breach  indicators  that  were  false. 
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Figure  12. 


Mean  Squared  Error  Results  for  Final  Duration  Forecast-Program  D 


Figure  13  shows  the  percent  difference  of  the  forecasted  duration  (IEAC(t))  from 
the  actual  duration  for  each  of  the  three  methods  plotted  over  time.  None  of  the  IEAC(t) 
methods  showed  a  high  confidence  in  helping  the  program  manager  forecast  events.  All 
IEAC(t)  method  values  were  at  least  30  percent  over  the  actual  duration  throughout  the 
majority  of  the  program.  Specifically,  forecast  methods  at  30  percent  different  from  the 
actual  duration  translate  into  missing  the  end  date  by  at  least  12  months.  That  type  of 
inaccuracy  in  the  forecast  values  would  immediately  lead  to  a  Nunn-McCurdy  Act 
schedule  breach  indicator.  History  shows  that  after  program  re-baseline  the  program  did 
not  experience  a  breach  and  completed  ahead  of  schedule.  The  data  showed  that 
acceptable  values,  less  than  14  percent  difference,  to  prevent  a  Nunn-McCurdy  breach 
indicator  were  generally  not  achieved  until  the  very  late  stages  of  the  program  where  it  is 
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easier  to  make  accurate  forecasts.  Though  ES  IEAC(t)  showed  improvement  over  the 
BCWS  and  EV  IEAC(t)  methods  would  still  not  give  a  program  manager  confidence  in 
forecasting  schedule  performance. 
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Figure  13.  Percent  Difference  Results  for  Final  Duration  Forecast-Program  D 


F.  U.S  ARMY  CMA  CASE  ANALYSIS  SUMMARY 

In  summary,  analysis  of  the  four  U.S.  Army  CMA  programs  showed  that  the  ES 
performance  metrics  were  more  accurate  indicators  of  schedule  performance  than  EVM 
metrics.  But,  they  still  did  not  forecast  schedule  accurately  enough  to  significantly 
improve  the  program  manager’s  ability  to  effectively  manage  schedule.  Each  of  the 
metrics  showed  IEAC(t)  values  that  exceeded  the  DoD  schedule  threshold  limits  on 
MDAPs  when  in  reality  neither  of  the  programs  had  a  schedule  breach  during  the 
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operations  phase.  The  IEAC(t)  methods  only  showed  accurate  values  during  the  late 
stage  program  stages  where  forecast  values  are  easier  to  predict. 

Table  3  shows  a  summary  of  ES  IEAC(t)  value  accuracy  compared  to  the  Nunn- 
McCurdy  Act  significant  breach  threshold. 


Table  3.  IEAC(t)  ES  Summary 


Program 

Total  Months 

Significant 

Variation  about  FD 

Breach 

Threshold 

(months) 

MSE 

%diff 

value 

months 

value 

months 

A 

51 

6 

110 

10 

25 

13 

B 

55 

6 

60 

8 

15 

9 

C 

52 

6 

57 

8 

30 

16 

D 

41 

6 

119 

10 

30 

11 
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IV.  SUMMARY  AND  RECOMMENDATIONS 


A.  SUMMARY 

This  Joint  Applied  Project  examined  DoD  EVM  methods  to  predict  schedule 
performance  and  final  program  duration  along  with  the  ES  method  to  predict  schedule 
performance  and  final  program  duration.  This  research  project  applied  the  schedule 
performance  metrics  to  four  U.S.  Anny  CMA  programs  to  determine  if  ES  metrics  were 
better  schedule  indicators  than  EV  metrics. 

The  research  showed  that  the  ES  metrics  were  generally  more  accurate  than  the 
EV  metrics  throughout  the  program  duration.  The  MSE  and  percent  difference  plots  of 
the  lEAC(t)  compared  to  the  actual  duration  showed  significantly  closer  forecast  values 
with  the  1EAC(J)es  method.  But,  the  ES  metrics  were  only  more  accurate  indicators 
relative  to  the  two  EV  metrics  and  did  not  forecast  duration  accurately  enough  to  be  a 
useful  program  management  tool.  ES  metrics  predicted  false  significant  schedule  breach 
indicators  on  each  of  the  four  programs,  which  would  have  incorrectly  caused  program 
stoppage,  corrective  actions,  and  potential  program  cancellation. 

The  current  DoD  EVM  policy  requirement  and  the  familiarity  of  EVM  tools  and 
techniques  make  the  relatively  new  and  unproven  ES  methods  difficult  to  implement  in 
DoD.  But,  this  research  showed  that  ES  performance  metrics,  specifically  the  ES  value, 
may  have  advantages  for  project  management  and  control  that  may  well  prove  beneficial 
for  DoD  programs.  Recommendations  are  provided  below  on  ways  that  DoD  can  use  ES 
methods  to  improve  schedule  performance  metrics  for  acquisition  programs. 

B.  RECOMMENDATIONS 

1.  Recommendation  #1 

Recommend  program  managers  apply  ES  performance  metrics  to  complement 
current  EVMS  metrics. 
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The  ES  metric  was  a  more  accurate  predictor  of  schedule  duration  compared  to 
traditional  EV  time  metrics  and  it  may  be  valuable  to  the  program  manager  tool  kit.  It 
was  easily  measured  from  CPR  EVM  cost  data  and  can  help  track  schedule  performance 
along  with  other  schedule  perfonnance  metrics.  The  author  believes  that  the  DoD 
acquisition  workforce  is  sufficiently  trained  in  EVM  principles  to  effectively  use  EVM 
data,  measure  the  ES  metric,  and  skillfully  interpret  the  results. 

2.  Recommendation  #2 

Recommend  that  ES  performance  metrics  for  IEAC(t)  do  not  replace  current  DoD 
EVMS  standards  for  schedule  performance  as  a  program  management  tool. 

ES  metrics  were  not  capable  of  forecasting  the  final  duration  within  a  sufficient 
level  of  confidence  for  a  program  manager.  Each  of  the  four  U.S.  Anny  CMA  programs 
showed  significant  Nunn-McCurdy  Act  schedule  breach  indicators  based  on  the  IEAC(t) 
ES  data  that  were  false.  A  program  manager  using  only  ES  perfonnance  metrics  would 
have  responded  incorrectly  to  false  forecast  final  duration  values. 

3.  Recommendation  #3 

Recommend  future  research  on  the  accuracy  of  ES  performance  methods 
compared  to  the  EVM  IMS  (DI-MGMT-81650,  2005)  and  critical  path  method  (CPM) 
schedule  metrics. 

CPM  is  a  program  scheduling  technique  that  aids  in  understanding  the 
dependency  of  events  in  a  program  and  the  time  required  to  complete  them.  It  uses  a 
logic  network  to  predict  final  project  duration  by  managing  those  sequences  of  activities 
necessary  to  complete  program  milestones.  IMS  is  an  integrated  schedule  that  details  the 
tasks  necessary  to  ensure  program  execution.  The  IMS  is  traceable  to  the  WBS  and  is 
used  to  measure  progress  toward  meeting  program  objectives  and  integrate  schedule 
activities  with  all  related  components.  Both  IMS  and  CPM  were  designed  to  track 
program  schedule  performance  and  a  comparative  study  between  them  and  ES  IEAC(t) 
would  show  potential  differences. 
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c. 


CONCLUSIONS 


In  conclusion,  this  Joint  Applied  Project  examined  DoD  EVM  equations  to 
predict  final  program  duration  and  the  ES  method  to  predict  final  program  duration.  ES 
schedule  performance  metrics  did  not  forecast  final  duration  accurately  enough  to 
warrant  using  them  as  a  program  management  tool  for  the  four  U.S.  Army  CMA 
programs. 

The  author  believes  that  some  aspects  of  ES  may  be  a  program  management  tool 
that  should  be  coupled  with  EVM  methods  to  more  completely  integrate  project  scope 
with  cost,  schedule,  and  performance  elements  for  optimum  project  planning  and  control. 
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